Cue-based audio coding/decoding

ABSTRACT

Generic and specific C-to-E binaural cue coding (BCC) schemes are described, including those in which one or more of the input channels are transmitted as unmodified channels that are not downmixed at the BCC encoder and not upmixed at the BCC decoder. The specific BCC schemes described include 5-to-2, 6-to-5, 7-to-5, 6.1-to-5.1, 7.1-to-5.1, and 6.2-to-5.1, where “0.1” indicates a single low-frequency effects (LFE) channel and “0.2” indicates two LFE channels.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.10/936,464 (“the '464 application”), filed on Sep. 8, 2004, which claimsthe benefit of the filing date of U.S. provisional application No.60/585,703, filed on Jul. 6, 2004, the teachings of which areincorporated herein by reference.

In addition, the '464 application is a continuation-in-part of thefollowing co-pending applications, the teachings of all of which areincorporated herein by reference:

-   -   U.S. application Ser. No. 09/848,877, filed on May 4, 2001;    -   U.S. application Ser. No. 10/045,458, filed on Nov. 7, 2001,        which itself claimed the benefit of the filing date of U.S.        provisional application No. 60/311,565, filed on Aug. 10, 2001;    -   U.S. application Ser. No. 10/155,437, filed on May 24, 2002;    -   U.S. application Ser. No. 10/246,570, filed on Sep. 18, 2002,        which itself claimed the benefit of the filing date of U.S.        provisional application No. 60/391,095, filed on Jun. 24, 2002;        and    -   U.S. application Ser. No. 10/815,591, filed on Apr. 1, 2004,        which itself claimed the benefit of the filing date of U.S.        provisional application No. 60/544,287, filed on Feb. 12, 2004.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to the encoding of audio signals and thesubsequent synthesis of auditory scenes from the encoded audio data.

2. Description of the Related Art

When a person hears an audio signal (i.e., sounds) generated by aparticular audio source, the audio signal will typically arrive at theperson's left and right ears at two different times and with twodifferent audio (e.g., decibel) levels, where those different times andlevels are functions of the differences in the paths through which theaudio signal travels to reach the left and right ears, respectively. Theperson's brain interprets these differences in time and level to givethe person the perception that the received audio signal is beinggenerated by an audio source located at a particular position (e.g.,direction and distance) relative to the person. An auditory scene is thenet effect of a person simultaneously hearing audio signals generated byone or more different audio sources located at one or more differentpositions relative to the person.

The existence of this processing by the brain can be used to synthesizeauditory scenes, where audio signals from one or more different audiosources are purposefully modified to generate left and right audiosignals that give the perception that the different audio sources arelocated at different positions relative to the listener.

FIG. 1 shows a high-level block diagram of conventional binaural signalsynthesizer 100, which converts a single audio source signal (e.g., amono signal) into the left and right audio signals of a binaural signal,where a binaural signal is defined to be the two signals received at theeardrums of a listener. In addition to the audio source signal,synthesizer 100 receives a set of spatial cues corresponding to thedesired position of the audio source relative to the listener. Intypical implementations, the set of spatial cues comprises aninter-channel level difference (ICLD) value (which identifies thedifference in audio level between the left and right audio signals asreceived at the left and right ears, respectively) and an inter-channeltime difference (ICTD) value (which identifies the difference in time ofarrival between the left and right audio signals as received at the leftand right ears, respectively). In addition or as an alternative, somesynthesis techniques involve the modeling of a direction-dependenttransfer function for sound from the signal source to the eardrums, alsoreferred to as the head-related transfer function (HRTF). See, e.g., J.Blauert, The Psychophysics of Human Sound Localization, MIT Press, 1983,the teachings of which are incorporated herein by reference.

Using binaural signal synthesizer 100 of FIG. 1, the mono audio signalgenerated by a single sound source can be processed such that, whenlistened to over headphones, the sound source is spatially placed byapplying an appropriate set of spatial cues (e.g., ICLD, ICTD, and/orHRTF) to generate the audio signal for each ear. See, e.g., D. R.Begault, 3-D Sound for Virtual Reality and Multimedia, Academic Press,Cambridge, Mass., 1994.

Binaural signal synthesizer 100 of FIG. 1 generates the simplest type ofauditory scenes: those having a single audio source positioned relativeto the listener. More complex auditory scenes comprising two or moreaudio sources located at different positions relative to the listenercan be generated using an auditory scene synthesizer that is essentiallyimplemented using multiple instances of binaural signal synthesizer,where each binaural signal synthesizer instance generates the binauralsignal corresponding to a different audio source. Since each differentaudio source has a different location relative to the listener, adifferent set of spatial cues is used to generate the binaural audiosignal for each different audio source.

SUMMARY OF THE INVENTION

In binaural cue coding (BCC), an encoder encodes C input audio channelsto generate E transmitted audio channels, where C>E≧1. In particular,two or more of the C input channels are provided in a frequency domain,and one or more cue codes are generated for each of one or moredifferent frequency bands in the two or more input channels in thefrequency domain. In addition, the C input channels are downmixed togenerate the E transmitted channels. In some downmixing implementations,at least one of the E transmitted channels is based on two or more ofthe C input channels, and at least one of the E transmitted channels isbased on only a single one of the C input channels.

In one embodiment, a BCC coder has two or more filter banks, a codeestimator, and a downmixer. The two or more filter banks convert two ormore of the C input channels from a time domain into a frequency domain.The code estimator generates one or more cue codes for each of one ormore different frequency bands in the two or more converted inputchannels. The downmixer downmixes the C input channels to generate the Etransmitted channels, where C>E≧1.

In BCC decoding, E transmitted audio channels are decoded to generate Cplayback audio channels. In particular, for each of one or moredifferent frequency bands, one or more of the E transmitted channels areupmixed in a frequency domain to generate two or more of the C playbackchannels in the frequency domain, where C>E≧1. One or more cue codes areapplied to each of the one or more different frequency bands in the twoor more playback channels in the frequency domain to generate two ormore modified channels, and the two or more modified channels areconverted from the frequency domain into a time domain. In some upmixingimplementations, at least one of the C playback channels is based on atleast one of the E transmitted channels and at least one cue code, andat least one of the C playback channels is based on only a single one ofthe E transmitted channels and independent of any cue codes.

In one embodiment, a BCC decoder has an upmixer, a synthesizer, and oneor more inverse filter banks. For each of one or more differentfrequency bands, the upmixer upmixes one or more of the E transmittedchannels in a frequency domain to generate two or more of the C playbackchannels in the frequency domain, where C>E≧1. The synthesizer appliesone or more cue codes to each of the one or more different frequencybands in the two or more playback channels in the frequency domain togenerate two or more modified channels. The one or more inverse filterbanks convert the two or more modified channels from the frequencydomain into a time domain.

BCC encoders and/or decoders may be incorporated into a number ofsystems or applications including, for example, digital videorecorders/players, digital audio recorders/players, computers, satellitetransmitters/receivers, cable transmitters/receivers, terrestrialbroadcast transmitters/receivers, home entertainment systems, and movietheater systems.

BRIEF DESCRIPTION OF THE DRAWINGS

Other aspects, features, and advantages of the present invention willbecome more fully apparent from the following detailed description, theappended claims, and the accompanying drawings in which like referencenumerals identify similar or identical elements.

FIG. 1 shows a high-level block diagram of conventional binaural signalsynthesizer;

FIG. 2 is a block diagram of a generic binaural cue coding (BCC) audioprocessing system;

FIG. 3 shows a block diagram of a downmixer that can used for thedownmixer of FIG. 2;

FIG. 4 shows a block diagram of a BCC synthesizer that can used for thedecoder of FIG. 2

FIG. 5 represents one possible implementation of 5-to-2 BCC processing;

FIG. 6 represents one possible implementation of 6-to-5 BCC processing;

FIG. 7 represents one possible implementation of 7-to-5 BCC processing;

FIG. 8 represents one possible implementation of 6.1-to-5.1 BCCprocessing;

FIG. 9 represents one possible implementation of 7.1-to-5.1 BCCprocessing; and

FIG. 10 represents one possible implementation of 6.2-to-5.1 BCCprocessing.

DETAILED DESCRIPTION

Generic BCC Processing

FIG. 2 is a block diagram of a generic binaural cue coding (BCC) audioprocessing system 200 comprising an encoder 202 and a decoder 204.Encoder 202 includes downmixer 206 and BCC estimator 208.

Downmixer 206 converts C input audio channels x_(i)(n) into Etransmitted audio channels y_(i)(n), where C>E≧1. In this specification,signals expressed using the variable n are time-domain signals, whilesignals expressed using the variable k are frequency-domain signals.Depending on the particular implementation, downmixing can beimplemented in either the time domain or the frequency domain. BCCestimator 208 generates BCC codes from the C input audio channels andtransmits those BCC codes as either in-band or out-of-band sideinformation relative to the E transmitted audio channels. Typical BCCcodes include one or more of inter-channel time difference (ICTD),inter-channel level difference (ICLD), and inter-channel correlation(ICC) data estimated between certain pairs of input channels as afunction of frequency and time. The particular implementation willdictate between which particular pairs of input channels, BCC codes areestimated.

ICC data corresponds to the coherence of a binaural signal, which isrelated to the perceived width of the audio source. The wider the audiosource, the lower the coherence between the left and right channels ofthe resulting binaural signal. For example, the coherence of thebinaural signal corresponding to an orchestra spread out over anauditorium stage is typically lower than the coherence of the binauralsignal corresponding to a single violin playing solo. In general, anaudio signal with lower coherence is usually perceived as more spreadout in auditory space. As such, ICC data is typically related to theapparent source width and degree of listener envelopment. See, e.g., J.Blauert, The Psychophysics of Human Sound Localization, MIT Press, 1983.

Depending on the particular application, the E transmitted audiochannels and corresponding BCC codes may be transmitted directly todecoder 204 or stored in some suitable type of storage device forsubsequent access by decoder 204. In either case, decoder 204 receivesthe transmitted audio channels and side information and performsupmixing and BCC synthesis using the BCC codes to convert the Etransmitted audio channels into more than E (typically, but notnecessarily, C) playback audio channels {circumflex over (x)}_(i)(n) foraudio playback. Depending on the particular implementation, upmixing canbe performed in either the time domain or the frequency domain.

In addition to the BCC processing shown in FIG. 2, a generic BCC audioprocessing system may include additional encoding and decoding stages tofurther compress the audio signals at the encoder and then decompressthe audio signals at the decoder, respectively. These audio codecs maybe based on conventional audio compression/decompression techniques suchas those based on pulse code modulation (PCM), differential PCM (DPCM),or adaptive DPCM (ADPCM).

Generic Downmixing

FIG. 3 shows a block diagram of a downmixer 300 that can used fordownmixer 206 of FIG. 2 according to certain implementations of BCCsystem 200. Downmixer 300 has a filter bank (FB) 302 for each inputchannel x_(i)(n), a downmixing block 304, an optional scaling/delayblock 306, and an inverse FB (IFB) 308 for each encoded channely_(i)(n).

Each filter bank 302 converts each frame (e.g., 20 msec) of acorresponding digital input channel x_(i)(n) in the time domain into aset of input coefficients {tilde over (x)}_(i)(k) in the frequencydomain. Downmixing block 304 downmixes each sub-band of C correspondinginput coefficients into a corresponding sub-band of E downmixedfrequency-domain coefficients. Equation (1) represents the downmixing ofthe kth sub-band of input coefficients ({tilde over (x)}₁(k),{tilde over(x)}₂(k), . . . , {tilde over (x)}_(C)(k)) to generate the kth sub-bandof downmixed coefficients (ŷ₁(k),ŷ₂(k), . . . , ŷ_(E)(k)) as follows:

$\begin{matrix}{{\begin{bmatrix}{{\hat{y}}_{1}(k)} \\{{\hat{y}}_{2}(k)} \\\vdots \\{{\hat{y}}_{E}(k)}\end{bmatrix} = {D_{CE}\begin{bmatrix}{{\overset{\sim}{x}}_{1}(k)} \\{{\overset{\sim}{x}}_{2}(k)} \\\vdots \\{{\overset{\sim}{x}}_{C}(k)}\end{bmatrix}}},} & (1)\end{matrix}$where D_(CE) is a real-valued C-by-E downmixing matrix.

Optional scaling/delay block 306 comprises a set of multipliers 310,each of which multiplies a corresponding downmixed coefficient ŷ_(i)(k)by a scaling factor e_(i)(k) to generate a corresponding scaledcoefficient {tilde over (y)}_(i)(k). The motivation for the scalingoperation is equivalent to equalization generalized for downmixing witharbitrary weighting factors for each channel. If the input channels areindependent, then the power p_({tilde over (y)}) _(i) _((k)) of thedownmixed signal in each sub-band is given by Equation (2) as follows:

$\begin{matrix}{{\begin{bmatrix}p_{{\overset{\sim}{y}}_{1}{(k)}} \\p_{{\overset{\sim}{y}}_{2}{(k)}} \\\vdots \\p_{{\overset{\sim}{y}}_{E}{(k)}}\end{bmatrix} = {{\overset{\_}{D}}_{CE}\begin{bmatrix}p_{{\overset{\sim}{x}}_{1}{(k)}} \\p_{{\overset{\sim}{x}}_{2}{(k)}} \\\vdots \\p_{{\overset{\sim}{x}}_{C}{(k)}}\end{bmatrix}}},} & (2)\end{matrix}$where D _(CE) is derived by squaring each matrix element in the C-by-Edownmixing matrix D_(CE).

If the sub-bands are not independent, then the power valuesp_({tilde over (y)}) _(i) _((k)) of the downmixed signal will be largeror smaller than that computed using Equation (2), due to signalamplifications or cancellations when signal components are in-phase orout-of-phase, respectively. To prevent this, the downmixing operation ofEquation (1) is applied in sub-bands followed by the scaling operationof multipliers 310. The scaling factors e_(i)(k) (1≦i≦E) can be derivedusing Equation (3) as follows:

$\begin{matrix}{{{e_{i}(k)} = \sqrt{\frac{p_{{\overset{\sim}{y}}_{i}{(k)}}}{p_{{\overset{\sim}{y}}_{i}{(k)}}}}},} & (3)\end{matrix}$where p_({tilde over (y)}) _(i) _((k)) is the sub-band power as computedby Equation (2), and p_(ŷ) _(i) _((k)) is power of the correspondingdownmixed sub-band signal ŷ_(i)(k).

In addition to or instead of providing optional scaling, scaling/delayblock 306 may optionally apply delays to the signals.

Each inverse filter bank 308 converts a set of corresponding scaledcoefficients {tilde over (y)}_(i)(k) in the frequency domain into aframe of a corresponding digital, transmitted channel y_(i)(n).

Although FIG. 3 shows all C of the input channels being converted intothe frequency domain for subsequent downmixing, in alternativeimplementations, one or more (but less than C−1) of the C input channelsmight bypass some or all of the processing shown in FIG. 3 and betransmitted as an equivalent number of unmodified audio channels.Depending on the particular implementation, these unmodified audiochannels might or might not be used by BCC estimator 208 of FIG. 2 ingenerating the transmitted BCC codes.

Generic BCC Synthesis

FIG. 4 shows a block diagram of a BCC synthesizer 400 that can used fordecoder 204 of FIG. 2 according to certain implementations of BCC system200. BCC synthesizer 400 has a filter bank 402 for each transmittedchannel y_(i)(n), an upmixing block 404, delays 406, multipliers 408,correlation block 410, and an inverse filter bank 412 for each playbackchannel {circumflex over (x)}_(i)(n).

Each filter bank 402 converts each frame of a corresponding digital,transmitted channel y_(i)(n) in the time domain into a set of inputcoefficients y_(i)(k) in the frequency domain. Upmixing block 404upmixes each sub-band of E corresponding transmitted-channelcoefficients into a corresponding sub-band of C upmixed frequency-domaincoefficients. Equation (4) represents the upmixing of the kth sub-bandof transmitted-channel coefficients ({tilde over (y)}₁(k),{tilde over(y)}₂(k), . . . , {tilde over (y)}_(E)(k)) to generate the kth sub-bandof upmixed coefficients ({tilde over (s)}₁(k),{tilde over (s)}₂(k), . .. , {tilde over (s)}_(C)(k)) as follows:

$\begin{matrix}{{\begin{bmatrix}{{\overset{\sim}{s}}_{1}(k)} \\{{\overset{\sim}{s}}_{2}(k)} \\\vdots \\{{\overset{\sim}{s}}_{C}(k)}\end{bmatrix} = {U_{EC}\begin{bmatrix}{{\overset{\sim}{y}}_{1}(k)} \\{{\overset{\sim}{y}}_{2}(k)} \\\vdots \\{{\overset{\sim}{y}}_{E}(k)}\end{bmatrix}}},} & (4)\end{matrix}$where U_(EC) is a real-valued E-by-C upmixing matrix. Performingupmixing in the frequency-domain enables upmixing to be appliedindividually in each different sub-band.

Each delay 406 applies a delay value d_(i)(k) based on a correspondingBCC code for ICTD data to ensure that the desired ICTD values appearbetween certain pairs of playback channels. Each multiplier 408 appliesa scaling factor a_(i)(k) based on a corresponding BCC code for ICLDdata to ensure that the desired ICLD values appear between certain pairsof playback channels. Correlation block 410 performs a matrix operationA based on corresponding BCC codes for ICC data to ensure that thedesired ICC values appear between certain pairs of playback channels.Further description of the operations of correlation block 410 can befound in U.S. patent application Ser. No. 10/155,437, filed on May 24,2002 as Baumgarte 2-10.

The synthesis of ICLD values may be less troublesome than the synthesisof ICTD and ICC values, since ICLD synthesis involves merely scaling ofsub-band signals. Since ICLD cues are the most commonly used directionalcues, it is usually more important that the ICLD values approximatethose of the original audio signal. As such, ICLD data might beestimated between all channel pairs. The scaling factors a_(i)(k)(1≦i≦C) for each sub-band are preferably chosen such that the sub-bandpower of each playback channel approximates the corresponding power ofthe original input audio channel.

One goal may be to apply relatively few signal modifications forsynthesizing ICTD and ICC values. As such, the BCC data might notinclude ICTD and ICC values for all channel pairs. In that case, BCCsynthesizer 400 would synthesize ICTD and ICC values only betweencertain channel pairs.

Each inverse filter bank 412 converts a set of corresponding synthesizedcoefficients

(k) in the frequency domain into a frame of a corresponding digital,playback channel {circumflex over (x)}_(i)(n).

Although FIG. 4 shows all E of the transmitted channels being convertedinto the frequency domain for subsequent upmixing and BCC processing, inalternative implementations, one or more (but not all) of the Etransmitted channels might bypass some or all of the processing shown inFIG. 4. For example, one or more of the transmitted channels may beunmodified channels that are not subjected to any upmixing. In additionto being one or more of the C playback channels, these unmodifiedchannels, in turn, might be, but do not have to be, used as referencechannels to which BCC processing is applied to synthesize one or more ofthe other playback channels. In either case, such unmodified channelsmay be subjected to delays to compensate for the processing timeinvolved in the upmixing and/or BCC processing used to generate the restof the playback channels.

Note that, although FIG. 4 shows C playback channels being synthesizedfrom E transmitted channels, where C was also the number of originalinput channels, BCC synthesis is not limited to that number of playbackchannels. In general, the number of playback channels can be any numberof channels, including numbers greater than or less than C and possiblyeven situations where the number of playback channels is equal to orless than the number of transmitted channels. For example, sixtransmitted, non-LFE channels be used to synthesize the six channels of5.1 surround sound, or vice versa.

5-to-2 BCC Processing

FIG. 5 represents one possible implementation of 5-to-2 BCC processingin which five input channels x_(i)(n) are downmixed to two transmittedchannels y_(i)(n), which are subsequently subjected to upmixing and BCCsynthesis to form five playback channels {circumflex over (x)}_(i)(n),for the loudspeaker arrangement shown in FIG. 5A.

In particular, FIG. 5A represents the downmixing scheme applied to thefive input channels x_(i)(n) to generate the two transmitted channelsy_(i)(n), where the downmixing matrix D₅₂ used in Equation (1) is givenby Equation (5) as follows:

$\begin{matrix}{{D_{52} = \begin{bmatrix}1 & 0 & \frac{1}{\sqrt{2}} & 1 & 0 \\0 & 1 & \frac{1}{\sqrt{2}} & 0 & 1\end{bmatrix}},} & (5)\end{matrix}$where the scale factors are chosen such that the sum of the squares ofthe value in each column is one, so that the power of each input signalcontributes equally to the downmixed signals. As shown in FIG. 5A,transmitted channel 1 (i.e., the left channel in a stereo signal) isgenerated from input channels 1, 3, and 4 (i.e., the left, center, andleft rear channels, respectively), and transmitted channel 2 (i.e., theright channel) is generated from input channels 2, 3, and 5 (i.e., theright, center, and right rear channels, respectively), where, accordingto Equation (5), the power of the center channel 3 is split evenlybetween the left and right transmitted channels.

FIG. 5B represents the upmixing scheme applied to the two transmittedchannels y_(i)(n) to generate five upmixed channels {tilde over(s)}_(i), where the upmixing matrix U₂₅ used in Equation (4) is given byEquation (6) as follows:

$\begin{matrix}{U_{25} = {\begin{bmatrix}1 & 0 \\0 & 1 \\1 & 1 \\1 & 0 \\0 & 1\end{bmatrix}.}} & (6)\end{matrix}$Note that, when the upmixed signals are subsequently normalized andre-scaled during ICLD synthesis, the scaling of the rows in the upmixingmatrix is not relevant. As shown in FIG. 5B, the transmitted leftchannel 1 is used as the base channel for the playback left and leftrear channels 1 and 4, the transmitted right channel 2 is used as thebase channel for the playback right and right rear channels 2 and 5, andthe sum of both transmitted channels is used as the base channel for theplayback center channel 3.

FIG. 5C shows the upmixing and BCC synthesis applied to the twotransmitted channels at the decoder. Note that ICTD and ICC synthesis isapplied between the channel pairs for which the same base channel isused, i.e., between the left and rear left channels and between theright and right rear channels. In one particular implementation of the5-to-2 BCC processing of FIG. 5, the BCC codes transmitted as sideinformation are limited to the ICLD values ΔL₁₂, ΔL₁₃, ΔL₁₄, and ΔL₁₅,the ICTD values τ₁₄ and τ₂₅, and the ICC values c₁₄ and c₂₅, where thesub-scripts identify the pair of channels between which the BCC codevalue is estimated. Other implementations can employ different sets ofBCC code data, including using a channel other than the left channel y₁as the reference for all ICLD estimates. In general, the transmitted BCCcode data can be limited to only those values needed to synthesize theplayback audio channels. Note that, in the implementation of FIG. 5C,ICTD and ICC synthesis is not applied to the center channel.

One advantage of the 5-to-2 BCC processing of FIG. 5 is that the twotransmitted channels 1 and 2 can be played back as left and rightchannels on a “legacy” stereo receiver that is unaware of BCC processingand ignores the BCC side information. The techniques applied in 5-to-2BCC processing can be generalized to any C-to-2 BCC scheme, where thetwo transmitted channels are capable of being played back on a legacystereo receiver. These techniques can be generalized further still toany C-to-E BCC scheme, where the E transmitted channels are capable ofbeing played back on a legacy E-channel receiver.

The 5-to-2 BCC scheme of FIG. 5 can also be extended to a 5.1-to-2 BCCscheme, where the six channels of 5.1 surround sound are downmixed totwo transmitted channels, where the “0.1” indicates the low-frequencyeffects (LFE) channel in 5.1 surround sound. In this scheme, like thecenter channel in FIG. 5, the LFE channel can be attenuated by 3 dB andadded to both transmitted channels. In that case, the base channel forsynthesizing the playback LFE channel at the decoder is the sum of thetwo transmitted channels, as is the case for the playback centerchannel. As described in U.S. patent application Ser. No. 10/827,900,filed on Apr. 20, 2004 , the teachings of which are incorporated hereinby reference, BCC processing for the LFE channel might only be appliedat certain (e.g., low) frequencies.

C-to-E BCC Processing with One or More Unmodified Channels

As mentioned earlier, in generating E transmitted channels from C inputchannels, one or more of the input channels may be transmitted asunmodified channels. In typical implementations, those unmodifiedchannels are not used to generate any downmixed channels nor any BCCcodes. Note that, in other possible implementations, in addition tobeing transmitted as unmodified channels, those input channels mightstill be used to generate one or more downmixed channels and/or some ofthe transmitted BCC codes. The following sections describe some possibleBCC schemes in which one or more of the input channels are transmittedunmodified.

As used in this specification, the term “unmodified” means that thecorresponding transmitted channel is based on only a single one of theinput channels. That is, the transmitted channel is not the result ofdownmixing two or more different input channels. Note that, although thechannel is referred to as being “unmodified,” it might nevertheless besubject to non-BCC audio codec processing, e.g., to reduce thetransmission bitrate.

6-to-5 BCC Processing

FIG. 6 represents one possible implementation of 6-to-5 BCC processingin which six input channels x_(i)(n) are downmixed to five transmittedchannels y_(i)(n), which are subsequently subjected to upmixing and BCCsynthesis to form six playback channels {circumflex over (x)}_(i)(n),for the loudspeaker arrangement shown in FIG. 6A. This 6-to-5 BCC schemecan be used for 5-channel backwards compatible coding of 6-channelsurround signals, such as those used in “Dolby Digital-Surround EX.”Inparticular, FIG. 6A represents the downmixing scheme applied to the sixinput channels x_(i)(n) to generate the five transmitted channelsy_(i)(n), where the downmixing matrix D₆₅ used in Equation (1) is givenby Equation (7) as follows:

$\begin{matrix}{{D_{65\;} = \begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 0 & \frac{1}{\sqrt{2}} \\0 & 0 & 0 & 0 & 1 & \frac{1}{\sqrt{2}}\end{bmatrix}},} & (7)\end{matrix}$where the three front channels 1, 2, and 3 are transmitted unmodifiedand the three rear channels 4, 5, and 6 are downmixed to two transmittedchannels 4 and 5, for a total of five transmitted channels. Thesix-loudspeaker setup shown in FIG. 6 is used in “Dolby Digital-SurroundEX.”

FIG. 6B represents the upmixing scheme applied to the five transmittedchannels y_(i)(n) to generate six upmixed channels {tilde over (s)}_(i),where the upmixing matrix U₅₆ used in Equation (4) is given by Equation(8) as follows:

$\begin{matrix}{U_{56} = {\begin{bmatrix}1 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 \\0 & 0 & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 1 \\0 & 0 & 0 & 1 & 1\end{bmatrix}.}} & (8)\end{matrix}$In this case, the base channels for playback channels 1, 2, 3, 4, and 5are the five transmitted channels 1, 2, 3, 4, and 5, respectively, whilethe sum of the left rear and right rear transmitted channels 4 and 5 isused as the base channel for the playback rear channel 6.

FIG. 6C shows the upmixing and BCC synthesis applied to the fivetransmitted channels at the decoder to generate the six playbackchannels. Since transmitted channels 1, 2, and 3 are unmodified, noforward and inverse filter banks are used for these channels, which aredelayed to compensate for the BCC processing time of the other channels.Moreover, in this implementation, no ICTD or ICC synthesis is applied togenerate playback channels 4, 5, and 6. As such, the BCC code data canbe limited to ΔL₄₆ and ΔL₅₆.

Another possibility for downmixing the six input channels would be toadd the left and rear left channels 1 and 4 to generate a firsttransmitted channel and add the right and rear right channels 2 and 5 togenerate a second transmitted channel, where the other two channels 3and 6 are left unmodified. In this 6-to-4 BCC scheme, the BCC synthesisat the decoder would apply ICTD and ICC synthesis between the playbackleft and rear left channels and between the playback right and rearright channels. Such a 6-to-4 BCC scheme would give more emphasis toleft/right independence, while the 6-to-5 BCC scheme of FIG. 6 givesmore emphasis to front/back independence.

7-to-5 BCC Processing

FIG. 7 represents one possible implementation of 7-to-5 BCC processingin which seven input channels x_(i)(n) are downmixed to five transmittedchannels y_(i)(n), which are subsequently subjected to upmixing and BCCsynthesis to form seven playback channels {circumflex over (x)}_(i)(n),for the loudspeaker arrangement shown in FIG. 7A. This 7-to-5 BCC schemecan be used for 5-channel backwards compatible coding of 7-channelsurround signals, such as those used in “Lexicon Logic 7.”

In particular, FIG. 7A represents the downmixing scheme applied to theseven input channels x_(i)(n) to generate the five transmitted channelsy_(i)(n), where the downmixing matrix D₇₅ used in Equation (1) is givenby Equation (9) as follows:

$\begin{matrix}{{D_{75} = \begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 1 & 0 & 1\end{bmatrix}},} & (9)\end{matrix}$where the three front channels 1, 2, and 3 are transmitted unmodified,and the four rear channels 4, 5, 6, and 7 are downmixed to twotransmitted channels 4 and 5, for a total of five transmitted channels.

FIG. 7B represents the upmixing scheme applied to the five transmittedchannels y_(i)(n) to generate seven upmixed channels {tilde over(s)}_(i), where the upmixing matrix U₅₇ used in Equation (4) is given byEquation (10) as follows:

$\begin{matrix}{U_{57} = {\begin{bmatrix}1 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 \\0 & 0 & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 1 \\0 & 0 & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 1\end{bmatrix}.}} & (10)\end{matrix}$In this case, the base channels for playback channels 1, 2, and 3 aretransmitted channels 1, 2, and 3, respectively, while transmittedchannel 4 is used as the base channel for both playback channels 4 and 6and transmitted channel 5 is used as the base channel for both playbackchannels 5 and 7.

FIG. 7C shows the upmixing and BCC synthesis applied to the fivetransmitted channels at the decoder to generate the seven playbackchannels. Since transmitted channels 1, 2, and 3 are unmodified, noforward and inverse filter banks are used for these channels, which aredelayed to compensate for the BCC processing time of the other channels.In this implementation, ICTD, ICLD, and ICC synthesis is applied betweenthe two playback rear left channels and between the two playback rearright channels. As such, the BCC code data can be limited to ΔL₄₆, ΔL₅₇,τ₄₆, τ₅₇, c₄₆, and c₅₇.

6.1-to-5.1 BCC Processing

FIG. 8 represents one possible implementation of 6.1-to-5.1 BCCprocessing in which seven input channels x_(i)(n) are downmixed to sixtransmitted channels y_(i)(n), which are subsequently subjected toupmixing and BCC synthesis to form seven playback channels {circumflexover (x)}_(i)(n), for the loudspeaker arrangement shown in FIG. 8A. This6.1-to-5.1 BCC scheme can be used for 5.1-surround backwards compatiblecoding of 6.1-surround signals, such as those used in “DolbyDigital-Surround EX.”

In particular, FIG. 8A represents the downmixing scheme applied to theseven input channels x_(i)(n) to generate the six transmitted channelsy_(i)(n), where the downmixing matrix D₇₆ used in Equation (1) is givenby Equation (11) as follows:

$\begin{matrix}{{D_{76} = \begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 1 & 0 & \frac{1}{\sqrt{2}} \\0 & 0 & 0 & 0 & 0 & 1 & \frac{1}{\sqrt{2}}\end{bmatrix}},} & (11)\end{matrix}$where the three front channels 1, 2, and 3 and the LFE channel 4 aretransmitted unmodified, and the three rear channels 5, 6, and 7 aredownmixed to two transmitted channels 5 and 6, for a total of sixtransmitted channels.

FIG. 8B represents the upmixing scheme applied to the six transmittedchannels y_(i)(n) to generate seven upmixed channels {tilde over(s)}_(i), where the upmixing matrix U₆₇ used in Equation (4) is given byEquation (12) as follows:

$\begin{matrix}{U_{67} = {\begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 0 & 0 \\0 & 0 & 0 & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 0 & 1 \\0 & 0 & 0 & 0 & 1 & 1\end{bmatrix}.}} & (12)\end{matrix}$In this case, the base channels for playback channels 1, 2, 3, 4, and 5are transmitted channels 1, 2, 3, 4, and 5, respectively, while the sumof the left rear and right rear transmitted channels 5 and 6 is used asthe base channel for the playback rear channel 7.

FIG. 8C shows the upmixing and BCC synthesis applied to the sixtransmitted channels at the decoder to generate the seven playbackchannels. Since transmitted channels 1, 2, 3, and 4 are unmodified, noforward and inverse filter banks are used for these channels, which aredelayed to compensate for the BCC processing time of the other channels.In this implementation, no ICTD or ICC synthesis is applied to generateplayback channels 5, 6, and 7. As such, the BCC code data can be limitedto ΔL₅₇ and ΔL₆₇.

Another possibility for downmixing the seven input channels would be toadd the left and rear left channels 1 and 5 to generate a firsttransmitted channel and add the right and rear right channels 2 and 6 togenerate a second transmitted channel, where the other three channels 3,4, and 7 are left unmodified. In this 6.1-to-4.1 BCC scheme, the BCCsynthesis at the decoder would apply ICTD and ICC synthesis between theplayback left and rear left channels and between the playback right andrear right channels. Such a 6.1-to-4.1 BCC scheme would give moreemphasis to left/right independence, while the 6. 1-to-5.1 BCC scheme ofFIG. 8 gives more emphasis to front/back independence.

7.1-to-5.1 BCC Processing

FIG. 9 represents one possible implementation of 7.1-to-5.1 BCCprocessing in which eight input channels x_(i)(n) are downmixed to sixtransmitted channels y_(i)(n), which are subsequently subjected toupmixing and BCC synthesis to form eight playback channels {circumflexover (x)}_(i)(n), for the loudspeaker arrangement shown in FIG. 9A. This7.1-to-5.1 BCC scheme can be used for 5.1-surround backwards compatiblecoding of 7.1-surround signals, such as those used in “Lexicon Logic 7Surround.”

In particular, FIG. 9A represents the downmixing scheme applied to theeight input channels x_(i)(n) to generate the six transmitted channelsy_(i)(n), where the downmixing matrix D₈₆ used in Equation (1) is givenby Equation (13) as follows:

$\begin{matrix}{{D_{86} = \begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 & 1 & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 0 & 1 & 0 & 1\end{bmatrix}},} & (13)\end{matrix}$where the three front channels 1, 2, and 3 and the LFE channel 4 aretransmitted unmodified, and the four rear channels 5, 6, 7, and 8 aredownmixed to two transmitted channels 5 and 6, for a total of sixtransmitted channels.

FIG. 9B represents the upmixing scheme applied to the six transmittedchannels y_(i)(n) to generate eight upmixed channels {tilde over(s)}_(i), where the upmixing matrix U₆₈ used in Equation (4) is given byEquation (14) as follows:

$\begin{matrix}{U_{68} = {\begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 0 & 0 \\0 & 0 & 0 & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 0 & 1 \\0 & 0 & 0 & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 0 & 1\end{bmatrix}.}} & (14)\end{matrix}$In this case, the base channels for playback channels 1, 2, 3, 4, and 5are transmitted channels 1, 2, 3, 4, and 5, respectively, whiletransmitted channel 5 is used as the base channel for both playbackchannels 5 and 7 and transmitted channel 6 is used as the base channelfor both playback channels 6 and 8.

FIG. 9C shows the upmixing and BCC synthesis applied to the sixtransmitted channels at the decoder to generate the eight playbackchannels. Since transmitted channels 1, 2, 3, and 4 are unmodified, noforward and inverse filter banks are used for these channels, which aredelayed to compensate for the BCC processing time of the other channels.In this implementation, ICTD, ICLD, and ICC synthesis is applied betweenthe two playback rear left channels and between the two playback rearright channels. As such, the BCC code data can be limited to ΔL₅₇, ΔL₆₈,τ₅₇, τ₆₈, c₅₇, and c₆₈.

6.2-to-5.1 BCC Processing

FIG. 10 represents one possible implementation of 6.2-to-5.1 BCCprocessing in which eight input channels x_(i)(n) are downmixed to sixtransmitted channels y_(i)(n), which are subsequently subjected toupmixing and BCC synthesis to form eight playback channels {circumflexover (x)}_(i)(n), for the loudspeaker arrangement shown in FIG. 10A,where the “0.2” denotes the presence of two LFE channels. This6.2-to-5.1 BCC scheme can be used for 5.1-surround backwards compatiblecoding of 6.2-surround signals.

In particular, FIG. 10A represents the downmixing scheme applied to theeight input channels x_(i)(n) to generate the six transmitted channelsy_(i)(n), where the downmixing matrix D₈₆ used in Equation (1) is givenby Equation (15) as follows:

$\begin{matrix}{{D_{86} = \begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 0 & 0 & 0 & 1 \\0 & 0 & 0 & 0 & 1 & 0 & \frac{1}{\sqrt{2}} & 0 \\0 & 0 & 0 & 0 & 0 & 1 & \frac{1}{\sqrt{2}} & 0\end{bmatrix}},} & (15)\end{matrix}$where the three front channels 1, 2, and 3 are transmitted unmodified,and the three rear channels 5, 6, and 7 and the two LFE channels 4 and 8are downmixed to three transmitted channels 4, 5, and 6, for a total ofsix transmitted channels.

FIG. 10B represents the upmixing scheme applied to the six transmittedchannels y_(i)(n) to generate eight upmixed channels {tilde over(s)}_(i), where the upmixing matrix U₆₈ used in Equation (4) is given byEquation (16) as follows:

$\begin{matrix}{U_{68} = {\begin{bmatrix}1 & 0 & 0 & 0 & 0 & 0 \\0 & 1 & 0 & 0 & 0 & 0 \\0 & 0 & 1 & 0 & 0 & 0 \\0 & 0 & 0 & 1 & 0 & 0 \\0 & 0 & 0 & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 0 & 1 \\0 & 0 & 0 & 0 & 1 & 1 \\0 & 0 & 0 & 1 & 0 & 0\end{bmatrix}.}} & (16)\end{matrix}$In this case, the base channels for playback channels 1, 2, 3, 5, and 6are transmitted channels 1, 2, 3, 5, and 6, respectively, whiletransmitted channel 4 is used as the base channel for both playback LFEchannels 4 and 8, and the sum of the left rear and right reartransmitted channels 5 and 6 is used as the base channel for theplayback rear channel 7.

FIG. 9C shows the upmixing and BCC synthesis applied to the sixtransmitted channels at the decoder to generate the eight playbackchannels. Since transmitted channels 1, 2, and 3 are unmodified, noforward and inverse filter banks are used for these channels, which aredelayed to compensate for the BCC processing time of the other channels.In this implementation, ICTD, ICLD, and ICC synthesis is applied betweenthe two playback LFE channels, but no ICTD or ICC synthesis is appliedto generate playback channels 5, 6, and 7. As such, the BCC code datacan be limited to ΔL₅₇, ΔL₆₇, ΔL₄₈, τ₄₈, and c₄₈.

Alternative Embodiments

BCC processing has been described in the context of generic as well as anumber of specific implementations. Those skilled in the art willunderstand that BCC processing can be extended to other specificimplementations involving just about any combination of any numbers ofnon-LFE channels and/or any numbers of LFE channels.

Although the present invention has been described in the context ofimplementations in which the encoder receives input audio signal in thetime domain and generates transmitted audio signals in the time domainand the decoder receives the transmitted audio signals in the timedomain and generates playback audio signals in the time domain, thepresent invention is not so limited. For example, in otherimplementations, any one or more of the input, transmitted, and playbackaudio signals could be represented in a frequency domain.

BCC encoders and/or decoders may be used in conjunction with orincorporated into a variety of different applications or systems,including systems for television or electronic music distribution, movietheaters, broadcasting, streaming, and/or reception. These includesystems for encoding/decoding transmissions via, for example,terrestrial, satellite, cable, internet, intranets, or physical media(e.g., compact discs, digital versatile discs, semiconductor chips, harddrives, memory cards, and the like). BCC encoders and/or decoders mayalso be employed in games and game systems, including, for example,interactive software products intended to interact with a user forentertainment (action, role play, strategy, adventure, simulations,racing, sports, arcade, card, and board games) and/or education that maybe published for multiple machines, platforms, or media. Further, BCCencoders and/or decoders may be incorporated in audio recorders/playersor CD-ROM/DVD systems. BCC encoders and/or decoders may also beincorporated into PC software applications that incorporate digitaldecoding (e.g., player, decoder) and software applications incorporatingdigital encoding capabilities (e.g., encoder, ripper, recoder, andjukebox).

The present invention may be implemented as circuit-based processes,including possible implementation as a single integrated circuit (suchas an ASIC or an FPGA), a multi-chip module, a single card, or amulti-card circuit pack. As would be apparent to one skilled in the art,various functions of circuit elements may also be implemented asprocessing steps in a software program. Such software may be employedin, for example, a digital signal processor, micro-controller, orgeneral-purpose computer.

The present invention can be embodied in the form of methods andapparatuses for practicing those methods. The present invention can alsobe embodied in the form of program code embodied in tangible media, suchas floppy diskettes, CD-ROMs, hard drives, or any other machine-readablestorage medium, wherein, when the program code is loaded into andexecuted by a machine, such as a computer, the machine becomes anapparatus for practicing the invention. The present invention can alsobe embodied in the form of program code, for example, whether stored ina storage medium, loaded into and/or executed by a machine, ortransmitted over some transmission medium or carrier, such as overelectrical wiring or cabling, through fiber optics, or viaelectromagnetic radiation, wherein, when the program code is loaded intoand executed by a machine, such as a computer, the machine becomes anapparatus for practicing the invention. When implemented on ageneral-purpose processor, the program code segments combine with theprocessor to provide a unique device that operates analogously tospecific logic circuits.

It will be further understood that various changes in the details,materials, and arrangements of the parts which have been described andillustrated in order to explain the nature of this invention may be madeby those skilled in the art without departing from the scope of theinvention as expressed in the following claims.

Although the steps in the following method claims, if any, are recitedin a particular sequence with corresponding labeling, unless the claimrecitations otherwise imply a particular sequence for implementing someor all of those steps, those steps are not necessarily intended to belimited to being implemented in that particular sequence.

1. A method for encoding C input audio channels to generate Etransmitted audio channels, the method comprising: providing two or moreof the C input channels in a frequency domain; generating one or morecue codes for each of one or more different frequency bands in the twoor more input channels in the frequency domain; and downmixing the Cinput channels to generate the E transmitted channels, where C>E≧1,wherein: at least one of the E transmitted channels is based on two ormore of the C input channels; and at least one of the E transmittedchannels is based on only a single one of the C input channels.
 2. Theinvention of claim 1, further comprising formatting the E transmittedchannels and the one or more cue codes into a transmission format suchthat: the format enables a first audio decoder having no knowledge ofthe existence of the one or more cue codes to generate E playback audiochannels based on the E transmitted channels and independent of the oneor more cue codes; and the format enables a second audio decoder havingknowledge of the existence of the one or more cue codes to generate morethan E playback audio channels based on the E transmitted channels andthe one or more cue codes.
 3. The invention of claim 2, wherein theformat enables the second audio decoder to generate C playback audiochannels based on the E transmitted channels and the one or more cuecodes.
 4. The invention of claim 1, wherein E=1.
 5. The invention ofclaim 1, wherein E>1.
 6. The invention of claim 1, wherein each of the Etransmitted channels is based on two or more of the C input channels. 7.The invention of claim 1, wherein: the C input channels comprise a firstlow frequency effects (LFE) channel; and at least one of the Etransmitted channels is based on only the first LFE channel.
 8. Theinvention of claim 1, wherein: the C input channels comprise at leasttwo LFE channels; and at least one of the E transmitted channels isbased on the at least two LFE channels.
 9. The invention of claim 1,wherein the one or more cue codes comprise one of more of inter-channellevel difference (ICLD) data, inter-channel time difference (ICTD) data,and inter-channel correlation (ICC) data.
 10. The invention of claim 1,wherein the downmixing comprises, for each of one or more differentfrequency bands, downmixing the two or more input channels in thefrequency domain into one or more downmixed channels in the frequencydomain.
 11. Apparatus for encoding C input audio channels to generate Etransmitted audio channels, the apparatus comprising: two or more filterbanks adapted to convert two or more of the C input channels from a timedomain into a frequency domain; a code estimator adapted to generate oneor more cue codes for each of one or more different frequency bands inthe two or more converted input channels; and a downmixer adapted todownmix the C input channels to generate the E transmitted channels,where C>E≧1, wherein: at least one of the E transmitted channels isbased on two or more of the C input channels; and at least one of the Etransmitted channels is based on only a single one of the C inputchannels.
 12. The invention of claim 11, wherein: the C input channelscomprise a first LFE channel; and at least one of the E transmittedchannels is based on only the first LFE channel.
 13. The invention ofclaim 11, wherein: the C input channels comprise at least two LFEchannels; and at least one of the E transmitted channels is based on theat least two LFE channels.
 14. The invention of claim 11, wherein: theapparatus is a system selected from the group consisting of a digitalvideo recorder, a digital audio recorder, a computer, a satellitetransmitter, a cable transmitter, a terrestrial broadcast transmitter, ahome entertainment system, and a movie theater system; and the systemcomprises the two or more filter banks, the code estimator, and thedownmixer.
 15. A method for decoding E transmitted audio channels togenerate C playback audio channels, the method comprising: upmixing, foreach of one or more different frequency bands, one or more of the Etransmitted channels in a frequency domain to generate two or more ofthe C playback channels in the frequency domain, where C>E≧1; applyingone or more cue codes to each of the one or more different frequencybands in the two or more playback channels in the frequency domain togenerate two or more modified channels; and converting the two or moremodified channels from the frequency domain into a time domain, wherein:at least one of the C playback channels is based on at least one of theE transmitted channels and at least one cue code; and at least one ofthe C playback channels is based on only a single one of the Etransmitted channels and independent of any cue codes.
 16. The inventionof claim 15, further comprising, prior to upmixing, converting the oneor more of the E transmitted channels from the time domain to thefrequency domain.
 17. The invention of claim 15, wherein E=1.
 18. Theinvention of claim 15, wherein E>1.
 19. The invention of claim 15,wherein each of the C playback channels is based on at least one of theE transmitted channels and at least one cue code.
 20. The invention ofclaim 15, wherein: the E transmitted channels comprise a first LFEchannel; and at least one of the C playback channels is based on onlythe first LFE channel and independent of any cue codes.
 21. Theinvention of claim 15, wherein: the E transmitted channels comprise afirst LFE channel; and at least two of the C playback channels are basedon the first LFE channel and at least one cue code.
 22. The invention ofclaim 15, wherein the one or more cue codes comprise one of more of ICLDdata, ICTD data, and ICC data.
 23. The invention of claim 15, whereinthe upmixing comprises, for each of one or more different frequencybands, upmixing at least two of the E transmitted channels into at leastone playback channel in the frequency domain.
 24. An apparatus fordecoding E transmitted audio channels to generate C playback audiochannels, the apparatus comprising: an upmixer adapted, for each of oneor more different frequency bands, to upmix one or more of the Etransmitted channels in a frequency domain to generate two or more ofthe C playback channels in the frequency domain, where C>E≧1; asynthesizer adapted to apply one or more cue codes to each of the one ormore different frequency bands in the two or more playback channels inthe frequency domain to generate two or more modified channels; and oneor more inverse filter banks adapted to convert the two or more modifiedchannels from the frequency domain into a time domain, wherein: at leastone of the C playback channels is based on at least one of the Etransmitted channels and at least one cue code; and at least one of theC playback channels is based on only a single one of the E transmittedchannels and independent of any cue codes.
 25. The invention of claim24, wherein: the E transmitted channels comprise a first LFE channel;and at least one of the C playback channels is based on only the firstLFE channel and independent of any cue codes.
 26. The invention of claim24, wherein: the E transmitted channels comprise a first LFE channel;and at least two of the C playback channels are based on the first LFEchannel and at least one cue code.
 27. The invention of claim 24,wherein: the apparatus is a system selected from the group consisting ofa digital video player, a digital audio player, a computer, a satellitereceiver, a cable receiver, a terrestrial broadcast receiver, a homeentertainment system, and a movie theater system; and the systemcomprises the upmixer, the synthesizer, and the one or more inversefilter banks.